Survey propagation: an algorithm for satisfiability 



A. Braunstein(^'^\ M. Mezard^^^ R. Zecchina^^^ 
^International School for Advanced Studies (SISSA), 
via Beirut 9, 34100 Trieste, Italy 
^ Laboratoire de Physique Theorique et Modeles Statistiques , 
CNRS and Universite Paris Sud, 
Bat. 100, 91405 Orsay cedex, Prance 
^ The Abdus Salam International Centre for Theoretical Physics (IGTP), 
Str. Costiera 11, 34 100 Trieste, Italy 



(Dated: 1st February 2008) 

We study the satisfiability of randomly generated formulas formed by M clauses of exactly K 
I literals over A*' Boolean variables. For a given value of the problem is known to be most difficult 

, when a = M/N is close to the experimental threshold separating the region where almost all 

' formulas are SAT from the region where all formulas are UNSAT. Recent results from a statistical 

04 ' physics analysis suggest that the difficulty is related to the existence of a clustering phenomenon 

^ , of the solutions when a is close to (but smaller than) Oc- We introduce a new type of message 

Oh' passing algorithm which allows to find efficiently a satisfiable assignment of the variables in this 

' difficult region. This algorithm is iterative and composed of two main parts. The first is a message- 

passing procedure which generalizes the usual methods like Sum-Product or Belief Propagation: it 
\l ■ passes messages that are surveys over clusters of the ordinary messages. The second part uses the 

' detailed probabilistic information obtained from the surveys in order to fix variables and simplify 

, the problem. Eventually, the simplified problem that remains is solved by a conventional heuristic. 

u: 

C/j ! I. INTRODUCTION 

The satisfiability problem is the archetype of combinatorial optimization problems which are well known to be 
''si" ' intractable in the worst case. However, experimental studies show that many instances of satisfiability are surprisingly 
^ , easy, even for naive heuristic algorithms. In an attempt to get a better understanding of which instances are easy or 
' hard to solve, a lot of eflForts have focused in recent years on the 'random K-sat' problem^. Instances of this problem 
are generated by considering N variables and M = aN clauses, where each clause contains exactly K distinct variables. 



o 

, and is picked up with uniform probability distribution from the set of f ^ j 2-^ possible clauses. For a given value 

' of a, the probability Pat (a) that a randomly generated instance is SAT is a decreasing function, with Pn{0) = 1 and 
I lima^oo Pn^o: = 0), which has been shown to approach, as A'' increases, a step function characteristic of a zero-one 
CZ2 . law 0, or a 'phase transition'. It is convenient to identify a crossover regime between 'SAT' and 'UNSAT' regimes 
J-J ' using the value ac{N) of the number of constraints per variable where PNiadN)) = 1/2 . From numerical simulations, 
ac{N) is supposed to converge, in the large N Hmit, to a value around ac — 4.27 0,0,0,0], but this convergence 
has not yet been established rigorously. Interestingly, the performance of algorithms is found to be much worse around 
this value of a = 4.27: randomly generated instances with a near to the phase transition are particularly difficult to 
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' solve. 



Rigorous lower and upper bounds have been found for this conjectured satisfiability threshold: it has been estab- 
lished that HniAT^tx) Pn{c<) = 1 for a < aih and Yiuin->oo Pn{c() = for a > aub- The present best bounds for the 
case of the random 3-SAT problem (with K ~ 3) are aub = 4.506 (from using the first moment method) and 
aib = 3.42 (from 0, using algorithmic analysis). Note also the interesting algorithm- independent upper bound found 
in 0,01 using the second moment method, which becomes better for larger values of K. 

Recently, some elaborate statistical physics methods have been brought to bear on the random satisfiability problem. 
These non-rigorous analytical calculations have put forward some interesting conjectures about what happens in the 
solution space of the problem as this threshold is approached (l^ (see also previous work in They suggest 

the following overall picture, which should hold for a generic sample of the random satisfiability problem, in the limit 
N — > 00, with a fixed. 

1. There exists a SAT-UNSAT phase transition at a critical value ac which can be computed by solving some 
(complicated) integral equation; for K = 3, one gets ac ~ 4.267 

2. There exists a second threshold adust separating two phases which are both 'SAT' (in each of them there exists 
a satisfiable assignment with probability 1), but with very different geometric structures: 
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3. For a < adust, a generic problem has many solutions, which tend to form one giant "cluster"; the set of all 
satisfying assignments forms a connected cluster in which it is possible to find a path between two solutions 
that requires short steps only (each pair of consecutive assignments in the path are close together in Hamming 
distance). In this regime, local search algorithms and other simple heuristics can relatively easily find a solution. 
This region is called the 'easy-SAT' region 

4. For adust < a < Uc, there exists a 'hard SAT' phase where the solution space breaks up into many smaller 
clusters. Solutions in separate clusters are generally far apart: it is not possible to transform a SAT assignment 
in one cluster into another one in a different cluster by changing only a finite number of variables. Because 
of this clustering effect, local search algorithms tend to have a very slow convergence when applied to large 
instances. 

So far, the analytic method used in the most recent statistical physics analysis, named the cavity metho dflnl. i s 
non rigorous, and turning this type of approach into a rigorous theory is an open subject of current research [13, 
Note however that, in the simpler case of the random K-XOR-SAT, the vaHdity of this statistical physics analysis can 
be confirmed by rigorous studies [13, [^, and the above clustering conjecture has been fully confirmed. 

Interestingly, the statistical physics analysis suggests a new efficient heuristic algorithm for finding SAT assignments 
in the hard SAT phase, which has been put forward by two of us in The aim of this paper is to provide a detailed 
self-contained description of this algorithm, which does not rely on the statistical physics background. We shall limit 
the description to the regime where solutions exist, the so called SAT phase; some modification of the algorithm allows 
to address the optimization problem of minimizing the number of violated constraints in the UNSAT phase, but it 
will not be discussed here. 

The basic building block of the algorithm, called survey propagation (SP), is a message passing procedure which 
resembles in some respect the iterative algorithm known as belief propagation (BP) , but with some crucial differences 
which will be described. BP is a generic algorithm for computing marginal probability distributions in problems 
defined on factor graphs, which has been very useful in the context of error correcting codes 0| and Bayesian 
networks (T^ . 

While in simple limits we are able to give some rigorous results together with an explicit comparison with the 
behef propagation procedures, in general there exists no rigorous proof of convergence of the algorithm. However, we 
provide clear numerical evidence of its performance over benchmarks problems which appear to be far larger than 
those which can be handled by present state-of-the-art algorithms. 

The paper is organized as follows: Sect. describes the satisfiability problem and its graphical representation in 
terms of a factor graph. Sect. IIIII explains two message passing algorithms, namely warning propagation (WP) and 
behef propagation (BP). Both are exact for tree factor graphs. Even if they are typically unable to find a solution 
for random SAT in the "interesting" hard-SAT region, they are shown here because they are in some sense the basic 
building blocks of our survey propagation algorithm. Sect. IIVI explains the survey propagation algorithm itself, a 
decimation procedure based on it, the 'survey inspired decimation' (SID), and the numerical results. In Sect^we give 
some heuristic arguments from statistical physics which may help the reader to understand where the SP algorithm 
comes from. Sect. EH contains a few general comments. 

II. THE SAT PROBLEM AND ITS FACTOR GRAPH REPRESENTATION 

We consider a satisfiability problem consisting of TV Boolean variables {xi £ {0, 1}} (where {0, 1} = {F, T}), with 
i S {1, ■■■,N}, with M constraints. Each constraint is a clause, which is the logical OR of the variables or of their 
negations. A clause a is characterized by the set of variables ii, ...,i_R- which it contains, and the fist of those which 
are negated, which can be characterized by a set of K numbers J° € {±1} as follows. The clause is written as 

(z,, V...Vz,^) (1) 

where Zi^ = xt^ if J" = — 1 and z^,. = .t,,, if Jf^ = 1 (note that a positive literal is represented by a negative J) . The 
problem is to find whether there exists an assignment of the xt G {0, 1} which is such that all the M clauses are true. 
We define the total cost C of a configuration x = (xi, ...,xn) as the number of violated clauses. 

In what follows we shall adopt the factor graph representation of the SAT problem. This representation is 
convenient because it provides an easy graphical description to the message passing procedures which we shall develop. 
It also applies to a wide variety of different combinatorial problems, thereby providing a unified notation. 

The SAT problem can be represented graphically as follows (see figOJ . Each of the N variables is associated to a 
vertex in the graph, called a "variable node" (circles in the graphical representation), and each of the M clauses is 
associated to another type of vertex in the graph, called a "function node" (squares in the graphical representation). 
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Figure 1: An example of a factor graph with 6 variable nodes i = 1, .., 6 and 6 function nodes a, b, c, d, e, f. The formula which 
is encoded is: F = {xi V xs) A {xi V 12 V X4) A (0:3 V xs) A {xs V 2:4 V 2:5) A {x2 V 0:4 V xe) A (15) 

A function node a is connected to a variable node i by an edge whenever the variable Xi (or its negation) appears 
in the clause a. In the graphical representation, we use a full line between a and i whenever the variable appearing 
in the clause is Xi (i.e. Jf = —1), a dashed line whenever the variable appearing in the clause is Xi (i.e. Jf = 1). 
Variable nodes compose the set X (\X\ — N) and function nodes the set A {\A\ = M). 

In summary, each SAT problem can be described by a bipartite graph, G — {X U A;E = X x A) where E is the 
edge set, and by the set of "coupHngs" {J^} needed to define each function node. For the K-SAT problem where each 
clause contains K variables, the degree of all the function nodes is K. 

Throughout this paper, the variable nodes indices are taken in i, j, fc, while the function nodes indices are taken 
in a, b, c, .... For every variable node i, we denote by V{i) the set of function nodes a to which it is connected by an 
edge, by rii = \V{i)\ the degree of the node, by V+(«) the subset of V{i) consisting of function nodes a where the 
variable appears un-negated (the edge (a,z) is a full line), and by V-{i) the complementary subset of V{i) consisting 
of function nodes a where the variable appears negated (the edge (a, i) is a dashed line). V{i) \ b denotes the set V(i) 
without a node b. Similarly, for each function node a, we denote by V{a) = V+(a) U V-.{a) the set of neighboring 
variable nodes, decomposed according to the type of edge connecting a and i, and by Ua the degree. Given a function 
node a and a variable node j, connected by an edge, it is also convenient to define the two sets: V"{j) and V^{j), 
where the indices s and u respectively refer to the neighbors which tend to make variable j satisfy or unsatisfy the 
clause a, defined as (see figEJ: 

if j; = 1 : K"(J) = V+{j) ; V:{j) = V-{j) \ a (2) 
if J; = -1: V:{j)^V^U) ; V:{j)^V+{j)\a (3) 

The same kind of factor graph representation can be used for other constraint satisfaction problems, where each 
function node a defines an arbitrary function over the set Xa C X of variable nodes to which is connected, and could 
also involve hidden variables. 



III. THE MESSAGE PASSING SOLUTION OF SAT ON A TREE 

In the special case in which the factor graph of a SAT problem is a tree (we shall call it a tree-problem), the 
satisfiability problem can be easily solved by many methods. Here we shall describe two message passing algorithms. 
The first one, called warning propagation (WP), determines whether a tree-problem is SAT or not; if it is SAT, 
WP finds one satisfiable assignment. The second algorithm, called belief propagation (BP), computes the number 
of satisfiable assignments, as well as the fraction of these assignments where a given variable is set to true. These 
algorithms are exact for tree-problems, but they can be used as heuristic in general problems, and we first give their 
general definition, which does not rely on the tree-like structure of the factor graph. 
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Figure 2: A function node a of the factor graph with the Va{j) and Vi{j) sets relative to node j. 



A. Warning propagation 



The basic elementary message passed from one function node a to a variable i (connected by an edge) is a Boolean 
number Ua^i G {0, 1} called a 'warning'. 

The update rule is defined as follows. Given a function node a and one of its variables nodes i, the warning Ua^i 
is determined from the warnings Ub^j arriving on all the variables j S V{a) \ i according to: 

]eVia)\i \ \beVij)\a J J 

where 0{x) = if a; < and 9{x) = 1 if a: > 0. This update rule is used sequentially, resulting in the following 
algorithm: 
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WP algorithm 

INPUT: the factor graph of a Boolean formula in conjunctive normal form; a maximal number of iterations tmax 
OUTPUT: UN-CONVERGED if WP has not converged after tmax sweeps. If it has converged: the set of all warnings 



0. At time t = 0: For every edge a ^ i of the factor graph, randomly initialize the warnings Ua^i{t = 0) G {0, 1}, e.g. 
with probability 1/2. 

1. For t = 1 to t = tmax- 

1.1 sweep the set of edges in a random order, and update sequentially the warnings on all the edges of the graph, 
generating the values Ua^i{t), using subroutine WP-UPDATE. 

1.2 If Ua~,i{t) = Ua~*iit — 1) on all the edges, the iteration has converged and generated u^^i = Ua^i{t): go to 2. 

2. If t = t„iax return UN-CONVERGED. If t < t„iax return the set of fixed point warnings = Ua^i{t) 



Subroutine WP-UPDATE(ua^«) 

INPUT: Set of all warnings arriving onto each variable node j e V{a) \ i 
OUTPUT: new value for the warning Ua^i- 

1 For every j e V{a) \ i, compute the cavity field hj^a = {j2bev+{j)\a ^''b~>j) - {'Ebev^{j)\a "f-^j) (If ^(j) \ a is 
empty, then hj^a = 0). 

2 Using these cavity fields hj^a, compute the warning Ua^i = Y[jev{a)\i^ i^j^aJj) (If ^(a) \ i is empty, then 

Ua^i = 1). 



The interpretation of the messages and the message-passing procedure is the following. A warning Ua^i — 1 can 
be interpreted as a message sent from function node a, teUing the variable i that it should adopt the correct value in 
order to satisfy clause a. This is decided by a according to the messages which it received from all the other variables 

j to which it is connected: if (j2i,^v{j}\a "^j^b-^j^ -^j < this means that the tendency for site j (in the absence of 
a) would be to take a value which does not satisfy clause a. If all neighbors j G V{a) \ i are in this situation, then a 
sends a warning to i. An example of the use of WP is shown in fig. El 

The warning propagation algorithm can be applied to any SAT problem. When it converges, this dynamics defines 
a fixed point, which is a set of warnings u*^,. These can be used to compute, for each variable i, the "local field" Hi 
and the "contradiction number" a which are two integers defined as: 



c» = 1 if E E > (6) 

\bev+{i) J V&ey-W / 

Ci = otherwise. (7) 

The local field Hi is an indication of the preferred state of the variable i: Xi = 1 if Hi > 0, Xi ^ if Hi < 0. The 
contradiction number indicates whether the variable i has received confiicting messages. 

The interest in WP largely comes from the fact that it gives the exact solution for tree-problems. This is summarized 
in the following simple theorem: 

THEOREM 1: 

Consider an instance of the SAT problem for which the factor graph is a tree. Then the WP algorithm converges to 
a unique set of fixed point warnings u^^^^, independently on the initial warnings. If at least one of the corresponding 
contradiction numbers Ci is equal to 1, the problem is UNSAT, otherwise it is SAT. 
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Corollary: In the case where the problem is SAT, the local fields Hi can be used to find an assignment of the 
variables satisfying all the clauses, using the following algorithm called "Warning Inspired Decimation" or WID: 

WID algorithm 

INPUT: the factor graph of a Boolean formula in conjunctive normal form 

OUTPUT: UN-CONVERGED, or status of the formula (SAT or UNSAT); If the formula is SAT: one assignment which 
satisfies all clauses. 



1. While the number of unfixed variables is > 0, do: 

1.1 Run WP 

1.2 If WP does not converge, return UN-CONVERGED. Else compute the local fields Hi and the contradiction 
numbers c,, using eqs. I|5I7|I . 

1.3 If there is at least one contradiction number c; = 1, return UNSAT. Else: 

1.3.1 If there is at least one local field Hi ^ 0: fix all variables with Hi ^ {Hi > ^ Xi = 1 and 
Hi < ^ Xi = ), and clean the graph, which means: { remove the clauses satisfied by this fixing, reduce 
the clauses that involve the fixed variable with opposite literal, update the number of unfixed variables}. 
GOTO label 1. Else: 

1.3.2 Choose one unfixed variable, fix it to an arbitrary value, clean the graph. GOTO label 1 

2. return the set of assignments for all the variables. 



PROOF of theorem 1: 

The convergence of message passing procedures on tree graphs is a well known result (see e.g.ji^). We give here 
an elementary proof of convergence for the specific case of WP, and then show how the results on Hi and Ci follow. 

Call £ the set of nodes. Define the leaves of the tree, as the nodes of degree 1. For any edge (a,i) connecting a 
function node a to a variable node i, define its level r as follows: remove the edge {a,i) and consider the remaining 
subgraph containing a. This subgraph Ta-i is a tree factor graph defining a new SAT problem. The level r is the 
maximal distance between a and all the leaves in the subgraph 7(a,i) (the distance between two nodes of the graph 
is the number of edges of the shortest path connecting them). If an edge (a,i) has level r = (which means that a 
is a leaf of the subgraph), Ua->i{t) = 1 for all t > 1. If (a,z) has level r = 1, then Ua^i{t) = for all t > 1. From 
the iteration rule, a warning Ua~,i at level r is fully determined from the knowledge of all the warnings Ub^j at levels 

< r — 2. Therefore the warning Ua^i{t) at a level r is guaranteed to take a fixed value for t > 1 + r/2. 
Let us now turn to the study of local fields and contradiction numbers. 

We first prove the following lemma: 
// a warning u*_fi = 1, the clause a is violated in the reduced SAT problem defined by the subgraph Ta^i- 
This is obviously true if the edge has level r = or r = 1. Supposing that it holds for all levels < r — 2, one considers 
an edge a — i aX level r, with u*^, = 1. From Q, this means that for all variable nodes j e V{a) \ i, the node j 
receives at least one message from a neighboring factor node b £ V{j) \ a with Ub^j = 1. The edge b — j is at level 

< r — 2, therefore the reduced problem on the graph Tb-j is UNSAT and therefore the clause b imposes the value of 
the variable j to be 1 (True) if Jj' = -1, or (False) if Jj = I: we shall say that clause b fixes the value of variable 
j. This is true for all j S V{a) \ i, which means that the reduced problem on ^(a,;) is UNSAT, or equivalently, the 
clause a fixes the value of variable i. 

Having shown that = 1 implies that clause a fixes the value of variable i, it is clear from ((JJ that a nonzero 

contradiction number Ci implies that the formula is UNSAT. 

If all the a vanish, the formula is SAT. One can prove this for instance by showing that the WID algorithm generates 
a SAT assignment. The variables with H receive some nonzero = 1 and are fixed. One then 'cleans' the 

graph, which means: remove the clauses satisfied by this fixing, reduce the clauses that involve the fixed variable with 
opposite literal. By definition, this process has removed from the graph all the edges on which there was a nonzero 
warning. So on the new graph, all the edges have u* — 0. Following the step 2.2 of WID, one chooses randomly a 
variable i, one fixes it to an arbitrary value Xi, and cleans the graph. The clauses a connected to i which are satisfied 
by the choice Xi are removed; the corresponding subgraphs are trees where all the edges have u* = 0. A clause a 
connected to i which are not satisfied by the choice Xi may send some u* = 1 messages (this happens if such a clause 
had degree 2 before fixing variable i). However, running WP on the corresponding subgraph T(^a,i}, the set of warnings 
can not have a contradiction: A variable j in this subgraph can receive at most one u* ~ 1 warning, coming from the 
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Figure 3: An example of result obtained by the WP algorithm on a tree-problem with N = 8 variables and AI — 9 clauses. 
The number on each edge of the graph is the value of the corresponding warning u* . The local fields on the variable are 
thus: 1,-1,1,2,0,0,0,0. The satisfiable assignments are such that xi = 1,X2 = 0,X3 = 1,xa = 1, x-j £ {0,1}, {x^^xa^xs) € 
{(1, 1, 1), (1, 1, 0), (0, 1, 1), (0, 0, 1)}. One can check that the variables with nonzero local field take the same value in all SAT 
assignments. In the WID algorithm, the variables 1,2,3,4 are fixed to xi = 1,X2 = 0,X3 = 1,X4 = 1; the remaining tree has 
only the clauses h and i remaining, and all the warnings are u* = 0. The variable x-j will be fixed arbitrarily. If one chooses 
for instance xs = 1, the remaining graph leaves xs unconstrained but imposes x^ = 1, as can be checked by applying to it the 
BP algorithm. 

unique path which connects j to a. Therefore cj = 0: one iteration of WID has generated a strictly smaller graph 
with no contradiction. By induction, it thus finds a SAT assignment. □ 

One should notice that the variables which are fixed at the first iteration of WID (those with non-zero Hi) are 
constrained to take the same value in all satisfiable assignments. 

B. Belief propagation 

While the WP algorithm is well adapted to finding a SAT assignment, the more compHcated belief propagation (BP) 
algorithm is able to compute, for satisfiable problems with a tree factor graph, the total number of SAT assignments, 
and the fraction of SAT assignments where a given variable Xi is true. 

We consider a satisfiable instance, and the probability space built by all SAT assignments taken with equal proba- 
bility. CaUing a one of the clauses in which Xi appears, the basic ingredients of BP are the messages: 

• j2a^i{xi) G [0, 1], interpreted as the probability that clause a is satisfied, given the value of the variable Xi € 
{0,1}. 

• fii^ai^i) G [0, 1], interpreted as the probability that the variable takes value xi, when clause a is absent (this 
is again a typical 'cavity' definition). Notice that J2xi£{o ly f^i^aixi) — 1, while there is no such normalization 

for ^Jia^i{Xi). 

The BP equations are: 

^li^a{Xi) = Ci^a W /ih^j(Xi) , (8) 
b£V('i)\a 

Ha^.,{Xi) = fa{X) Jl lJ-j^a{Xj) , (9) 

where C,;_,a is a normalization constant ensuring that fJ-i^a is a probability, the sum over {xj{j ^ i)\ means a sum 
over all values of the variables Xj G {0, 1}, for all j different from i, and fa{X) is a characteristic function taking value 
1 if the configuration X = {xi} satisfies clause a, taking value otherwise. 
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It is convenient to parameterize ^i^a{xi) by introducing the number 7i_,a G [0, 1] which is the probabiUty that the 
variable Xi is in the state which violates clause a, in a problem where clause a would be absent (writing for instance 
fj,t^aixi) = 'yi-,aS{xi,0) + (1 - ji-,a)S{xi, 1) in the case where J° = -1). 

Let us denote by 

]eVia)\i 

the probability that all variables in clause a, except variable i, are in the state which violates the clause. 
The BP algorithm amounts to an iterative update of the messages Sa^i according to the rule: 

BP algorithm: INPUT: the factor graph of a Boolean formula in conjunctive normal form; a maximal number of iterations 
tmax\ a requested precision e. 

OUTPUT: UN-CONVERGED if BP has not converged after tmax sweeps. If it has converged: the set of all messages <5*^j. 



0. At time t = 0: For every edge a ^ i of the factor graph, randomly initialize the messages Sa^i{t = 0) G [0, 1] 

1. For t = 1 to t = tmax- 

1.1 sweep the set of edges in a random order, and update sequentially the warnings on all the edges of the graph, 
generating the values 5a^i{t), using subroutine BP-UPDATE. 

1.2 If \Sa^i{t) — Sa~*i{t — 1)1 < e on all the edges, the iteration has converged and generated S*^^ ~ 5a^i{t): go 
to 2. 

2. If t = t„iax return UN-CONVERGED. If t < tmax return the set of fixed point warnings 5a ^ i* = Sa~,i{t) 



Subroutine BP-UPDATE((5a^,) 

INPUT: Set of all messages arriving onto each variable node j G V{a) \ i 
OUTPUT: new value for the message Sa-ti- 



1 For every j G V{a) \ i, compute the cavity field 

pu 

7j-a = (11) 

j^a ^ j^a 

where 

P^^a = n (l--^"-.) ■ (12) 

If an ensemble is empty, for instance V^{j) = 0, the corresponding -PjLa takes value 1 by definition. 

2 Using these numbers Jj^a, compute the new message: Sa^i = Yijev(a)\i^3^a,- If a factor node a is a leaf (unit 
clause) with a single neighbor i, the corresponding Sa^i takes value 1 by definition. 



As WP, the BP algorithm is exact on trees (see for instance 113 )■ In f^ct it gives a more accurate results than WP 
since it allows to compute the exact probabilities m (while WP identifies the variables which are fully constrained, 
and gives a zero local field on the other variables). A working example of BP is shown in figEl In this example 
and more in general for trees, BP also provides the exact number J\f of SAT assignments, as given by the following 
theorem: 

THEOREM 2: Consider an instance of the SAT problem for which the factor graph is a tree, and there exist 
some SAT assignments. Then: 
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Figure 4: An example of result obtained by the BP algorithm on the tree-problem with A*' = 8 variables and M — 9 clauses 
studied in figOl On each edge of the graph connecting a function node like c to a variable node like 2, appears the value of 72-.C 
(in blue, on the right hand side of the edge) and of (5c^2 (in red, on the left hand side of the edge). Comparing to the WP result 
of figEl one sees that all the messages 5a~ti = 1, corresponding to strict warnings, are the same, while the messages 5a~*i < 1 
are interpreted in WP as "no warning" (i.e. a null message). Using l|13|l . the probability Hi that each variable Xi = 1 are found 
equal to: 1, 0, 1, 1, 1/2, 3/4, 3/4, 1/2. These are the exact results as can be checked by considering all satisfiable assignments as 
in figH 



a) The BP algorithm converges to a unique set of fixed point messages S*^^ 

b) The probability /Xi that the variable Xi = 1 is given by: 



c) The number of SAT assignments is N — cxp(S'), where the entropy S is given by: 



(13) 



S = ^In 



n n (1 



iex 



n (1 

b£V+(i) 



n (1- 

beV^ii) 

+ n (1 

bev^(i) 



n n (1 

iev{a) Xbev^ii) 



(14) 



PROOF: 

The proof of convergence is simple, using the same strategy as the proof in sect lllll messages at level and 1 are 
fixed automatically, and a message at level r is fixed by the values of messages at lower levels. 

The probability fii is computed from the same procedure as the one giving the BP equations H8I9|I , with the difference 
that one takes into account all neighbors of the site i. 

The slightly more involved result is the one concerning the entropy. We use the probability measure P{X) on 
the space of all assignments which has uniform probability for all SAT assignments and zero probability for all the 
assignments which violate at least one clause: 



(15) 



aeA 



From P one can define the following marginals: 

• the 'site marginal' Pi{xi) is the probability that variable i takes value Xj G {0, 1} 

• the 'clause marginal' Pa{Xa) is the probability that the set of variables Xi,i £ V{a), takes a given value, denoted 
by Xa (among the 2"" possible values). 
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For a tree factor graph, one easily shows by induction on the size of the graph that the full probability can be expressed 
in terms of the site and clause marginals as: 



The entropy S — ln(7V) is then obtained as 

Let us now derive the expression of this quantity in terms of the messages used in BP. One has: 

b<£V{2) 



and 



ieV{a) 



where Ci and Ca are two normalization constants. From itTsf one gets after some reshuffling: 



i Xi i a i^V{a) Xa 



beV{i}\a 



Using the BP equation Q, this gives: 



a Xa 



Y[ Hi-,a{xi)fa{Xa) 
ieV{a) 



(16) 



(17) 



(18) 



(19) 



(20) 



-E E 1^^^— (21) 

a ieV{a) 



where the term ,fa{Xa) inside the logarithm has been added, taking into account the fact that, as fa{Xa) G {0, 1}, 
one always has Pa{Xa) \nfa{Xa) = 0. Therefore: 



a i a ieV{a) 



In the notations of (1T2I1. one has 



(22) 



(23) 



pti I ps 
i — >a ' i — >a 



and 



Substitution into Il22|l gives the expression Ijl4ll for the entropy. □ 



(24) 



(25) 



IV. SURVEY PROPAGATION 



A. The algorithm 



The WP and BP algorithms have been shown to work for satisfiability problems where the factor graph is a tree. 
In more general cases where the factor graph has loops, they can be tried as heuristics, but there is no guarantee 



11 



of convergence. In this section we present a new message passing algorithm, survey propagation (SP), which is also 
a heuristic, without any guarantee of convergence. It reduces to WP for tree-problems, but it turns out to be more 
efficient than WP or BP in experimental studies of random satisfiability problems. SP has been discovered using 
concepts developed in statistical physics under the name of 'cavity method'. Here we shall first present SP, then give 
some experimental results, and in the end expose the qualitative physical reasoning behind it. 

A message of SP, called a survey, passed from one function node a to a variable i (connected by an edge) is a real 
number i]a-^i G [0, 1]. The SP algorithm uses exactly the same main procedure as BP fsee llll BI . but instead of caUing 
the BP-UPDATE, it uses a different update rule, SP-UPDATE, defined as: 

Subroutine SP-UPDATE(77a^j) 

INPUT: Set of all messages arriving onto each variable node j G V{a) \ i 
OUTPUT: new value for the message ija-ti- 



1 For every j G V{a) \ i, compute the three numbers: 



n" 



n° 



1- n i^-vb^i) 

bev^-U) 

n (1-%-.) 

b€V{j)\a 



n i^-Vb^l) 

bev^'U) 



if a set like V^{j) is empty, the corresponding product takes value 1 by definition. 
2 Using these numbers , compute and return the new survey: 

jeV{a)\i 



(26) 



(27) 



If V{a) \ i \s empty, then rja^i = 1. 



Qualitatively, the statistical physics interpretation of the survey ?7a-»i is a probability that a warning is sent from a 
to i (see section^for details). Therefore, whenever the SP algorithm converges to a fixed-point set of messages ?7a_»i, 
one can use it in a decimation procedure in order to find a satisfiable assignment, if such an assignment exists. This 
procedure, called the survey inspired decimation (SID), is a generalization of the WID algorithm IIII Al defined by: 



SID algorithm 

INPUT: The factor graph of a Boolean formula in conjunctive normal form. A maximal number of iterations t,nax and a 
precision e used in SP 

OUTPUT: One assignment which satisfies all clauses, or 'SP UNCONVERGED', or 'probably UNSAT' 



0. Random initial condition for the surveys 

1. Run SP. If SP does not converge, return 'SP UNCONVERGED' and stop (or restart, i.e. go to 0.). If SP 
converges, use the fixed-point surveys J/^^j in order to: 

2. Decimate: 

2.1 If non-trivial surveys {{rj ^ 0}) are found, then: 
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(a) Evaluate, for each variable node i, the three 'biases' {wj;^\w^ \w^^^}def\ned by: 



(+) 



n+ + nr + no 

nr 

1 - - w'>~^ 



(28) 

(29) 
(30) 



where n+,n,- are defined by 

n+ = 
nr = 



1- n (I-'?- 

aeV+(i) 



1 n (1 

aeV-(i) 

n 



n (1 

a£V-(i) 

n (1 

oey+(i) 



Va- 



(31) 



(b) fix the variable with the largest 



(+) 



w} 'l to the value x,. 



1 if W^i"*"^ >M^i to the value a;, 



if < . Clean the graph, which means: { remove the clauses satisfied by this fixing, reduce the 

clauses that involve the fixed variable with opposite literal, update the number of unfixed variables}. 

2.2 If all surveys are trivial ({rj = 0}) , then output the simplified sub-formula and run on it a local search 
process (e.g. walksat). 

4. If the problem is solved completely by unit clause propagation, then output "SAT" and stop. If no contradiction is 
found then continue the decimation process on the smaller problem (go to 1.) else (if a contradiction is reached) 
stop. 



There exist several variants of this algorithm. In the code which is available at [22||, for performance reasons 
we update simultaneously all rj belonging to the same clause. The clauses to be updated are chosen in a random 
permutation order at each iteration step. The algorithm can also be randomized by fixing, instead of the most biased 
variables, one variable randomly chosen in the set of the x percent variables with the largest bias. This strategy 
allows to use some restart in the case where the algorithm has not found a solution. A fastest decimation can also be 
obtained by fixing in the step 2.1(b), instead of one variable, a fraction / of the Nt variables which have not yet been 
fixed (going back to 1 variable when fNt < 1). 



B. Experimental study of the SP algorithm 

In order to get some concrete information on the behaviour of SP for large but finite N, we have experimented SP 
and SID on single instances of the random 3-SAT problem with many variables, up to TV ~ 10''. In this section we 
summarize these (single machine) experiments and their results. 

Instances of the 3-SAT problem were generated with the pseudo random number generator "Algorithm B" on p. 32 
of Knuth [2^. However we found that results are stable with respect to changes in the random number generators. 
Formulas are generated by choosing k-tuples of variable indices at random (with no repetitions) and by negating 
variables with probability 0.5. 

We first discuss the behaviour of the SP algorithm itself. We have used a precision parameter e = lO^'^ (smaller 
values don't seem to increase performance significantly). Depending on the range of a, we have found the following 
behaviours, for large enough N: 

• For a < ad ^ 3.9, SP converges towards the set of trivial messages r/a-ti = 0, for all a — i edges. All variables 
are under-constrained. 
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• For 3.9 < a < 4.3, SP converges to a unique fixed-point set of non-trivial messages, independently from the 
initial conditions, where a large fraction of the messages rja^i are in ]0, 1[. 

Notice that, for 'small' values of N, around N = 1000, one often finds some instances in which SP does not 
converge. But the probability of convergence, at a given a < 4.3, increases with N. This is exemplified by the 
following quantitative measure of the performance of the SID algorithm (which uses SP). We have solved several 
instances of the random 3-SAT problem, for various values of a and A^, using the SID algorithm in which we fix at 

each step the fraction fNt of variables with largest W^~^^ — ^ . Table IIV Bl gives in each case the fraction of 

samples which are solved by SID, in a single run of decimation (without any restart). The algorithm fails when, either 
SP does not converge, or the simplified sub-formula found by SID is not solved by walksat. The performance of SID 
improves when increases and when / decreases. Notice that for N = 10^ we solve all the 50 randomly generated 
instances at a = 4.24. For larger values of a the algorithm often fails. Notice that in such cases it does not give 
any information on whether the instance is UNSAT. Some failures may be due to UNSAT instances, others are just 
real failures of the SID for SAT instances. Few experiments on even larger instances (A^ = 10^, 10^) have also been 
succesfully run (the limiting factor being the available computer memory needed to store formulas). 

As shown by the data on the average total number of SP iterations along the successful solution process in table 



IIV Bl the convergence time of the SP algorithm basically does not grow with A^ (a growth like log A^, which could be 
expected from the geometrical properties of the factor graph, is not excluded). Therefore the process of computing all 
the SP messages ?7*^j takes Q{N), or maybe Q{NlnN), operations. If SID fixes at each step only one variable, it will 
thus converge in 9(A^^ log A^) operations (the time taken by walksat to solve the simplified sub-formula seems to grow 
more slowly). When we fix a fraction of variables at a time, we get a further reduction of the cost to 0(A^(ln A^)^) 
(the second In comes from sorting the biases). 

A very basic yet complete version of the code which is intended to serve only for the study on random 3-SAT 
instances is available at the web site [2^ . GeneraHzation of the algorithm to other problems require some changes 
which are not implemented in the distributed code. 



V. HEURISTIC ARGUMENTS 



Survey propagation has been invented using powerful concepts and methods developed in the statistical physics of 
disordered systems, notably the cavity method for diluted problems In this section we want to give some short 
background on these methods, in order to help the reader understand where SP comes from, and maybe develop 
similar algorithms in other contexts. Unfortunately so far there is no rigorous derivation of the cavity method, so this 
whole section only contains heuristic arguments. 
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Figure 5: Results obtained by solving with a single decimation run of the SID algorithm 50 random instances of 3-SAT 
with N = 25000,50000, 100000 and a = 4.21,4.22,4.23,4.24. SID was used by fixing variables in blocks fNt, where Nt is 
the number of unfixed variables at time t, and various runs used different values of / taken in the geometric progression 
/ = 4%, 2%, 1%, .5%, .25%, .125%, stopping if the formula was solved. For each value of (A, a, /), we give the fraction of the 
50 instances which were solved (i.e. for which the algorithm found a SAT assignement). The maximal number of iteration was 
taken equal to 10^ and the precision for convergence was taken equal to 10~^. The last row shows the number of complete 
iterations of SP averaged over the successful runs. 
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A. The physical picture underlying the SP construction: clustering of configurations 

Let us start with a discussion of the validity of BP. As we saw in sectHni BP aims at computing the marginal 
probability distribution of a variable Xi, within the probability space built by all SAT assignments, each being given 
equal probability. The message fia^i{xi) used in BP can be computed exactly if one knows the joint probability 
distribution P^°'^{X) of the variables in X — {xj, j G V{a) \ i}, in the graph where clause a is absent. Using the 
same notations as in one has: 

|Ia-.^{x^)= fa{X)P^''\X) (32) 

Comparing this eq. Ij32|) to the eqs. H8I9|I of BP, one sees that BP uses an approximation, namely the fact that the 
joint probability P'-^'^X) factorizes: P'-'^^X) ~ 0, 

&v{a)\i f^j^aixj). This amounts to assuming that the variables 
Xj, for j e V{a) \ i, are uncorrelated in the absence of clause a. This assumption is obviously correct when the factor 
graph is a tree, which confirms the validity of BP in that case. In a general problem, for such an assumption to hold, 
we need two conditions to be fulfilled: 

• the variables Xj,j € V{a) \ i should be far from each other, in the factor graph where clause a is absent. 

• there should be a single phase in the problem, or the probability measure should be reduced to one single pure 
phase. 

The first condition is easily understood, and one can see that it is generically fulfilled when one considers a random 
satisfiability problem. In random K-sat with M = aN clauses, the factor graph is a random bipartite graph, where 
function nodes have degree K and variable nodes have fiuctuating degrees, with a distribution which becomes, in the 
large N limit, a Poisson law of mean Ka. Locally such a graph is tree like. Taking a clause a at random, the minimal 
distance between two of the variables Xj,j € V{a) \i is generically of order log A^. 

The second condition is more subtle. In general a pure phase is defined in statistical physics as an extremal Gibbs 
measure however the standard construction of Gibbs measures deals with infinite systems. Here we need to work 
with N variables where A^ ^ 1 but is finite, and the corresponding construction has not been worked out yet. For 
the random satisfiability problem, a heuristic description of a pure phase is a cluster of SAT assignments, defined as 
follows. Consider all SAT assignments. Define the distance between two assignments {xi} and {yi} as J^ii^i ~ Vi)'^- 
If the distance between two SAT assignment is smaller than a number q, they are said to belong to the same g-cluster. 
This allows to partition the set of SAT assignments into g-clusters. One is interested in the 'clusters' obtained in 
large N - large q limit of the g-clusters, where the large N limit is taken first. Heuristic statistical physics arguments 
indicate that, in the random K-satisfiability problem, for a < aciust{K), there should exist one single such cluster 
of SAT assignments: this means that one can move inside the space of SAT assignment, from any assignment to 
another one, by a succession of moves involving each a number of fiips of variables which is << N. In such a case the 
BP factorization approximation is expected to be correct. On the other hand, for a > aciust[K) the space of SAT 
assignment separates into many distant clusters, and the BP factorization does not hold globally, but it would hold 
if one could restrict the probability space to one given cluster a. Within such a restricted space, BP would converge 
to a set of messages ^"^^(xi) which depends on the cluster a. 

In this situation, the cavity method uses a statistical approach. It considers all the clusters of SAT assignments, 
and attributes to each cluster a probability proportional to the number of configuration that it contains. Then one 
introduces, on each edge a — i, a. survey which gives the probability, when a cluster a is chosen randomly with this 
probability, that the message /i"_^j(a;) is equal to a certain function P{x). 

This object is a probability of a probability and it is thus difficult to use in practical algorithms. For this reason, 
SP departs from the usual cavity method and uses a simpler object which is a survey of warnings, interpreted as 
follows: Consider one cluster a and an edge a — i of the factor graph. If, in every SAT assignments of the cluster 
a, all the variables Xj,j G V{a) \ i don't satisfy clause a, then a warning = 1 is passed along the edge from 

a to i. The SP message along this edge is the survey of these warnings, when one picks up a cluster a at random: 
77a->i = '""-♦^/(X^q basically the SP message gives the probability that there is a warning sent from a to i 

in a randomly chosen cluster. With respect to the full-fiedged cavity method, this is a much simplified object, which 
focuses onto the variables which are constrained. 

The experimental results on random 3-satisfiability discussed in sect. IIVBI confirm the theoretical analysis of 
[T^ IT3 | which indicate that all rja-,i vanish for a < ad — 3.91. This can be interpreted as the fact that, in this 
range of a, there are no 'constrained clusters' (meaning clusters in which some of the variables are constrained). For 
ad < a < ttc = 4.267 the theory predicts the existence of non-trivial messages, meaning that there exist constrained 
clusters. This is the region where SP and SID are able to outperform existing algorithms. One should notice that the 
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SID does not seem to converge up to the conjectured threshold Uc = 4.267: although it is very small, there remains a 
region in the 'Hard SAT' phase, close to ac where the simple version of SP/SID does not give the result. Recent work 
shows that this gap can be reduced by using some backtracking strategy in SID[23|- Whether it will be possible to 
close the gap with generalized versions of SP/SID, while keeping a typically polynomial running time, is an interesting 
open issue. 



B. The "don't-care" state 



In a given cluster a, a variable Xi can be thought of being in three possible states: either it is constrained equal 
to (this means that Xi = in all SAT assignments of the cluster a), or it is constrained equal to 1, or it is not 
constrained. In this last situation we attribute it the value Therefore we can describe a cluster by the values of the 
N generaHzed variables xf S {0, 1, *}, where * will be denoted as the don't-care state. Such a description associates 
to each cluster a single point in {0, 1, *}^. It discards a lot of information: a;f = * has lost all the information on 
the fraction of assignments in the cluster a where Xi = Q. But it gives a simplified description of the cluster and it 
focuses onto the constrained variables. 

It is interesting to notice that the SP equations can be interpreted as BP equations in the presence of this extra 
don't-care state. This can be seen as follows. Borrowing the notations of the BP equations (|HI El we denote by 
7i^a e [0, 1] the probability that the variable Xi is in the state which violates clause a, in a problem where clause a 
would be absent, and by 



]£V{a)\i 



(33) 



the probability that all variables in clause a, except variable i, are in the state which violates the clause. 

Let us compute ^i^a{xi). This depends on the messages sent from the nodes b £ {V{i) \ a} to variable i. The 
various possibilities for these messages are: 



• No warning arriving from b & Vi^{i), and no warning arriving from b € This happens with a probability 

^° - " Sh^,) (34) 



beV{i)\a 



No warning arriving from 6 £ V^{i), and at least one warning arriving from b £ V^{i). This happens with a 
probability 



1- n (1-5,^,) 



n i^-st^'O 



(35) 



• No warning arriving from b G V^{i), and at least one warning arriving from b G V^{i). This happens with a 
probability 



1- n (1-'^" 



n i^-Sb^^) 

bev^'ii) 



(36) 



• At least one warning arriving from b € V^{i), and at least one warning arriving from b € V^{i). This happens 
with a probability 



1- n (1-'^' 

bev^(t) 



b—tij 



1- n (1-'^''-) 

bev^{i) 



(37) 



As we work only with SAT configurations, the contradictory messages must be excluded. Therefore, the probability 
7j-»a G [0, 1] that the variable Xi is in the state which violates clause a, given that there is no contradiction, is: 



(38) 



The above equations in the enlarged space including the null message, given in (I33I38|I are identical to the SP 
equations Ij27l26ll . with the identification rja^i = 6a^i. 



16 



3.9 3.95 



4.05 4.1 4.15 4.2 4.25 4.3 

a 



Figure 6: Complexity per variable (E/A^) of the satisfying clusters for a given sample of size A'^ = 10®. The complexity vanishes 
at Qc which is expected to be the critical threshold for the specific formula under study. 



C. Complexity 



The above interpretation of SP using the don't-care state suggest a method to estimate the complexity, defined 
as the normaHzed logarithm of the number of constrained clusters of SAT assignments. As each constrained cluster 
of SAT assignments is associated with a point in {0,1,*}^, the complexity should be given by the correspon ding 
entropy, which can be estimated with the usual BP formula JEl in the presence of the don't-care state (see ref. 
for a rigorous derivation). The result, originally derived in from a direct statistical physics analysis without the 
use of the don't-care state, is as follows. 

The total complexity S can be decomposed into contributions associated with every function node and with every 
variable, and reads: 



M 



N 



o=l 



i=l 



(39) 



where 



log 



n (n^ 

j£V{a) 



n n^.. 



(40) 



S,. = +log 



ny 



(41) 



In Fig. El we report the data for the complexity of random 3-SAT formulae of size N = 10^ and a in the clustering 
range. This has been obtained with the following procedure. One generates first a 'random' 3-SAT formula with 
N = 10^ and M = 4.27 10^, using a pseudo-random number generator. The SP algorithm is run on this formula, and 
the complexity is evaluated from ll39)l . Then a new formula is generated from the previous one by ehminating lO'^ 
(pseudo-)randomly chosen clauses, and the algorithm is run again, etc... It turns out that, for N = 10^, the resulting 
curve is very 'reproducible', in the sense that the fiuctuations of the curve from one random instance to the next are 
small (typically below 1%). 



D. Interpretation of the SID algorithm: categories of variables 

Once SP has reached convergence, we can compute the total biases Wj^°^}. According to the previous discus- 

sion, these numbers should be interpreted as giving the fraction of constrained clusters where the variable Xi is respec- 
tively (frozen positive)/ (frozen negative)/ unconstrained. Having computed these weights, we may distinguish three 
reference types of variable nodes (of course all the intermediate cases will also be present): the under-constrained 
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Figure 7: A simple SAT problem with three variables, two 2-clauses, one 3-clause, and loops. Each figure gives one of the two 
possible solutions of warning propagation. The number on each edge (a, i) is the warning Ua^i, the number next to each variable 
node is the corresponding generalized state of the variable in each of the two 'clusters'. The SP equations for this graph have 
an infinite number of solutions, with -qa^i = r]b^2 ~ x, ria^2 ~ Vt^i = y, Vc^i = 7?c^2 = 0, and rjc^s = (1 — x)^y'^/{l — xy)^ 



ones with W> the biased ones with either ^ ~ 1 or and the balanced ones with Wl^' ~ Wf 

and H^/"^ small. 

Fixing a variable of each of these types produces different effects, consistently with the interpretation of the surveys. 
The following behaviours can be easily checked in numerical experiments: 

Fixing a biased variable does not alter the structure of the clusters and the complexity changes smoothly (few 
constrained clusters are eliminated). This is the strategy used by the SID procedure. 

Fixing an under-constrained variable has no effect on the complexity. 

As expected, fixing a balanced variable produces a decrease very close to In 2 in the complexity. 



E. A summary of the main conjectures 



The whole interpretation relies on the existence, in certain 'hard SAT' regions of parameters (here in a window of 
a below etc), of clusters of SAT assignments, which are very far apart (one cannot reach a cluster from another one 
unless one flips a flnite fraction of the TV variables). It would be very interesting to prove this statement. 

When there exist such clusters, some of them may be 'constrained clusters', in which some variables are constrained 
to take the same value for all the assignments of the cluster. The introduction of the don't-care state is an attempt at 
identifying all these clusters (in which each cluster is characterized by a single point in the enlarged assignment space 
including the don't-care state). SP, interpreted as BP in this enlarged assignment space, is an attempt at obtaining 
a statistical description of these constrained clusters. 

Two conjectures arise naturahy from the heuristic statistical physics approach and the numerical experiments on 
SP. They should hold for the random satisflability problem, in the large TV hmit, in some hard SAT window of a just 
below Qfc [3^ : 

• 1) With probability one (on the set of initial messages), SP converges to a unique set of fixed point messages. 

• 2) These fixed point messages contain the correct information about the constrained clusters, and in particular 
the number of constrained clusters can be computed as in (I39|l . 

Note that for finite N, there are obvious counterexamples to these conjectures, as shown in fig. [3 



VI. COMMENTS AND PERSPECTIVES 



When the solutions of a SAT (or more generahy of a constrained satisfaction problem) tend to cluster into well 
separated regions of the assignment space, it often becomes difficult to find a global solution because (at least in ah 
local methods) , different parts of the problem tend to adopt locally optimal configurations corresponding to different 
clusters, and these cannot be merged to find a global solution. In SP we proceed in two steps. First we define some 
elementary messages which are warnings, characteristic of each cluster, then we use as the main message the surveys 
of these warnings. The warning that we used is a rather simplified object: it states which variables are constrained 
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and which are not constrained. Because of this simpUfication the surveys can be handled easily, which makes the 
algorithm rather fast. As we saw in sect0 one might also aim at a finer description of each cluster, where the 
elementary messages would give the probability that a variable is in a given state (£ {0, 1}), when considering the set 
of all SAT configurations inside this given cluster. In this case the survey will give the probability distribution of this 
probability distribution, when one chooses a cluster randomly. This is more cumbersome algorithmically, but there is 
no difficulty of principle in developing this extension. One could also think of generaHzing further the messages (to 
probability distributions of probability distributions of probability distributions) , if some problems have a structure 
of clusters within other clusters (this is known to happen for instance in spin glasses ^^), but the cost in terms of 
computer resources necessary to implement it might become prohibitive. 

We have presented here a description of a new algorithmic strategy to handle the SAT problem. This is a rather 
general strategy which can also be applied in principle, to all Constraint Satisfaction Problems 0, 01 ■ At the 
moment the approach is very heuristic, although it is based on a rather detailed conjecture concerning the phase 
structure of random 3-SAT. The validity of a similar conjecture has been checked exactly in the random XOR-SAT 
problem. It would be interesting to study this algorithmic strategy in its own, independently from the random 3- 
SAT problem and the statistical physics study. One can expect progress on SP to be made in the future in various 
directions, among which: Rigorous results on convergence, different ways of using the information contained in the 
surveys, generalization of the algorithm to deal with complex graphs which are not typical random graphs, use of 
the generaHzed SP with penalty to provide UNSAT certificates. The generaHzation of SP to generic Constraint 
Satisfaction Problems is discussed in a separate pubHcation [3^ . 
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